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A  basic  assumption  of  least-squares  regression  theory  Is  that  the 
regressors  are  uncorrelated  with  the  disturbance.   '«fhen  this  assumption 
falls,  so  does  the  consistency  of  least  squares.   Such  a  failure  can 
typlcallv  be  regarded  as  a  specification  error  and  many  such  errors  can 
be  cast  into  this  form.   Failure  may  Involve,  for  example,  the  omission 

of  regressor-correlated  variables  from  the  equation  or  measurement 

2 
error  in  the  regressors  or  simply  the  treating  of  endogenous  variables 

as  exogenous  through  the  neglect  of  simultaneous  feedback  effects. 

Although  the  results  of  this  paper  are  stated  in  terms  of  least  squares, 
they  obviously  apply  to  anv  estimator  which  can  be  put  in  a  least-squares- 
like  form,  hence  to  two-stage  least  squares,  instrumental  variables,  and 
the  like.   It  does  not  seem  worthwhile  making  such  obvious  generality 
explicit  in  the  notation  or  the  text. 

2 

This  is  the  case  treated  in  M.  D.  Levi,  "Errors  in  the  Variables  Bias  in 

the  Presence  of  Correctly  Measured  Variables,"  Econometrlca,  Al,  September, 

1973,  pp.  985-86.   Levi  considers  the  case  of  only  one  variable  measured 

with  error  (which  is  one  way  of  regarding  the  problem  treated  in  the  present 
paper)  and  obtains  the  result  as  to  the  direct  effect  given  below,  showing 

also  that  the  signs  of  the  indirect  effects  depend  on  the  cofactors  of  the 

cross-product  matrix  of  regressors.       (v)  "13  iS"!"-! 
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It  sometimes  happens  that  the  crucial  assumption  can  be  maintained 
with  respect  to  most  of  the  regressors  but  fails  as  regards  one  of  them. 

Thus,  a  single  variable  may  be  measured  with  error  or  a  single  variable 
involved  in  a  simultaneous  feedback  with  all  other  variables  safely 
exogenous.   In  such  a  circumstance,  the  effect  of  the  error  on  the 
estimated  coefficient  of  the  affected  variable  would  often  be  easy  to 
analyze,  at  least  as  to  sign,  were  that  variable  the  only  one  in  the 
regression.   In  essence,  one  simply  examines  the  sources  of  the  problem 
to  determine  the  sign  of  the  correlation  between  the  disturbance  and  the 
variable  in  question.   The  effect  of  the  error  on  the  estimated  coef- 
ficient will  be  in  the  same  direction. 

Where  the  affected  variable  is  not  the  only  regressor,  however,  the 
situation  is  not  quite  so  clear.   If  the  remaining,  "unaffected"  variables 
are  not  all  orthogonal  to  the  affected  one,  will  the  sign  of  the  direct 
effect  remain  as  in  the  simple  case?  Moreover,  it  is  an  elementary  error 
to  suppose  that  the  effects  of  the  specification  error  are  confined  to 
the  coefficient  of  the  directly  affected  variable.   In  general,  there 
will  be  indirect  effects  as  well,  and  various  conjectures  arise  as  to 
them. 

Thus,  for  example,  one  might  suppose  that  fef ter  adjusting  for  the 
units  in  which  the  variables  are  measured)  the  direct  effect  will  be 
larger  than  the  Indirect  effects.   One  might  also  suppose  that  a  particular 
indirect  effect  will  be  larger,  the  larger  is  the  correlation  between  the 
unaffected  variable  in  question  and  the  directly  affected  variable. 

In  the  present  paper,  I  examine  these  and  related  questions.   I  show 
that  while  the  sign  of  the  direct  effect  does  Indeed  remain  the  same  as 
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In  the  simple  case,  the  conjectures  as  to  the  indirect  effects  are  wrong 
unless  there  is  only  one  unaffected  regressor  (apart  from  the  constant 
term)  or  unless  all  unaffected  regressors  are  mutually  orthogonal.   In 
particular,  It  is  possible  to  produce  an  example  in  which  the  ratio  of 
every  Indirect  effect  to  the  direct  effect  can  be  made  indefinitely  large. 
Thus,  not  only  is  it  false  to  suppose  that  the  effects  of  simple  specifi- 
cation error  are  confined  to  the  coefficient  of  the  directly  affected 
variable,  it  is  wrong  to  suppose  that  such  coefficient  suffers  from  the 
principal  effect. 

Fortunately,  the  signs  of  the  indirect  effects  can  be  estimated 
from  the  data,  as  can  the  ratios  of  the  Indirect  effects  to  the  direct 
effect,  so  that  the  analyst  need  not  throw  up  his  hands.   Indeed,  most 
of  the  results  are  obtained  by  showing  that  the  ratios  in  question  are 
given  by  the  negative  of  the  vector  of  regression  coefficients  when  the 
affected  variable  is  regressed  on  the  unaffected  ones. 

I  now  proceed  to  the  formal  analysis.   In  the  usual  notation,  the 
equation  to  be  estimated  is: 

(1)  Y  =  X6  +  e, 

where  Y  is  a  T-vector  of  observations  on  the  dependent  variable,  X  is  a 
T  X  k  matrix  of  observations  on  k  regressors,  S  is  a  k-vector  of  parameters 
to  be  estimated,  and  e  is  a  T-vector  of  disturbances.   I  assume  that 
Qi  Pllm  (X'X/T)  exists  and  is  non-singular  and  that: 

(2)  Pllm  (^)   =  X  ej^   , 
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where  X  is  a  non-zero  scalar  and  e  is  the  k-vector  with  last  component 

unity  and  remaining  components  zero.   Thus  the  crucial  assumption  of 

least  squares  is  violated  only  for  X  ,  the  last  regressor.   The  sign 

of  the  correlation  between  X,  and  e  is  that  of  A . 

k 

In  view  of  (2),  least  squares  will  not  be  consistent,  and,  by 
Theil's  specification  error  theorem,  the  inconsistency,  denoted  by  6, 
will  be  given  by: 

(3)       6  =  Plim  (X'X)"-Sc'Y  -  6  =  Plim  (X'X)"-^X'e  =  X  Q'^e      . 

..-   I  have  already  implicitly  denoted  the  j  th  column  of  X  by  X  .   Now 
denote  by  X(j),  the  matrix  formed  from  X  by  deleting  the  j th  column. 
Define: 

(A)       Y=  Plim  ({X(k)'X(k)}"-'-X(k)'Xj^)    "  ■' 

so  that  Y  Is  asymptotically  the  vector  of  regression  coefficients  when 

the  directly  affected  variable,  X,  ,  is  regressed  on  the  remaining  variables, 

I  now  prove: 


Theorem:       (a)  X  5j^  >  0   ;    (b)  ^./^^   =  -Y.   ,  j  ?*  k. 


Proof:         (a)  From  (3),  5   =  ^(Q""*")!^},^  an«^  (Q  ^)j,^^  >   0.  since  Q  is 

positive  definite. 

(b)  From  (3),  6     =  (Q   )   .   Hence  6/6   =  Q  -^ /Q   ,  where 
J        k]  J   k 

Q   denotes  the  cof actor  of  Q , . •   Without  loss  of  generality  suppose  that 

I  work  in  terms  of  inconsistency  rather  than  bias  principally  because  the 
results  as  to  indirect  effects  involve  ratios.   Some  of  the  results  would 
hold  for  bias  under  appropriate  assum.ptions. 


J  -  1.   Then 
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(5) 


,kl 
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(-l)-'-"^S)et  Pllm  {X(k)'X(l)/T} 
(-l)^*^  Det  Pllm  {X(k)'X(k)/T} 


(-l)-'-"^^  Plim  {Det  (X(k)'X(k))  ■'^X(k) 'X(l) } 


(-1) 
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where  the  last  equality  follows  from  the  fact  that  the  first  k-1  columns 
of  X(l)  are  identical  to  the  last  k-1  columns  of  X(k).   Expanding  by 
cofactors  of  the  first  row  yields 


(6) 


h^\ 


(_,)l+k^_^^l.(k-l) 


-^1  ' 


and  the  theorem  is  proved. 

Thus  the  direct  effect  is  the  same  as  in  the  simple  case,  while 
the  ratios  of  the  indirect  effects  to  the  direct  effect  are  given  by  the 
negative  of  the  vector  of  regression  coefficients  when  the  affected 
variable  is  regressed  on  the  remaining  variables.   We  now  go  on  to  use 
these  results  in  considering  the  conjectures  raised  above. 

I  begin  by  considering  the  relative  sizes  of  the  indirect  and  direct 
effects.  Clearly  such  relative  size  depends  in  the  first  instance  on  the 
units  in  which  the  different  variables  are  measured.   This  is  evident  either 
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dlrectly  from  the  fact  that  6./6,  Is  not  unit  free  or  from  the  fact 

that  Y.  Is  not.   Hence,  by  choosing  units  in  which  the  variance  of  X^ 

is  made  very  small  relative  to  the  variance  of  any  X  ,  j  tj*  k,  we  can 

make  the  ratios  of  indirect  to  direct  effects  as  large  as  we  like 

(unless  X  is  orthogonal  to  all  the  other  X  ).   This  does  not  say  very 
k  J 

much. 

If  the  conjecture  that  direct  effects  are  likely  to  be  larger  than 
indirect  effects  has  any  content,  therefore,  it  must  involve  the  case  in 
which  units  are  chosen  to  take  out  such  trivial  scale  effects.   Accordingly, 
I  now  assume  that  all  variables  are  measured  from  their  means  and  divided 
by  their  standard  deviations;  hence,  all  variables  now  have  mean  zero 
and  variance  unity  and  the  constant  term  is  suppressed.   I  shall  call 
these  "standard  units".  Q  now  becomes  a  matrix  of  correlation  coefficients, 
with 

the  correlation  between  variables  1  and  j. 
It  is  now  easy  to  prove. 


Corollary:     In  standard  units,  if  either  k  =  2  or  r   =  0  for  all 


i,  j,  k?«i?tj9«k,  then 


6./<5,   <  1  . 


Proof:         In  either  of  the  two  cases,  y     will  be  the  same  as  the 
regression  coefficient  in  the  simple  regression  of  X,  on  X  .   With  the 
units  chosen  as  described,  however,  this  is  r,  . . 

Unfortunately,  no  similar  result  holds  in  more  general  cases.   To 
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see  this,  it  suffices  to  examine  a  three-variable  case  with  r   ?*  0. 
Thus,  choose  r^.  "  V  ~   1»  with  p  >  0.   Choose  r^-  =  r-»  =  r  >  0  and 
r  «  h  /p/2   ,  where  0  <  h  <  1.   Positive  definiteness  of  Q  requires 
y  <  2  and  Det  Q  >  0.   Examining  the  latter  condition,  we  see  that; 

(8)        .  .  Det  Q  =  1  +  2(y  -  l)r^  -  2r^  -    (v   -   D^ 

-   (2r^  -  u)(u  -  2)  -  u(h^  -  l)(p  -  2)  >  0 


in  view  of  the  restrictions  placed  on  h  and  y. 

From  the  theorem,  however,  we  need  only  examine  Yi  and  y_.   In  this 
case, 


(9)     -y^  =  -y^  =  lSM::lhL    „  r(uz2)_  .  _  r/,  =  - 


l-(y-l)      2u-u  /  2y 

The  absolute  value  of  which  tends  to  infinity  as  y  tends  to  zero.  Hence 
every  indirect  effect  can  be  indefinitely  large  relative  to  the  direct 
effect.        .      .    . 

It  is  clear  from  the  corollary  and  the  above  example  that  colinearity 
among  the  unaffected  variables  can  produce  relatively  big  indirect  effects. 
However,  it  is  wrong  to  conclude  that  such  colinearity  has  a  monotonic 
effect.   Consider  the  general  three  variable  case  (still  in  standard 
units).  Then: 


(10)  6^/63  =  -Y^  -   ^^  ^^  2     ^^ 

^  -  ^12 
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If  r  _  and  r   have  the  same  sign,  for  example,  an  increase  in  r 
from  zero  can  change  the  sign  of  y-,  .  and  therefore,  of  6   as  well 
(since  6.  >  0,  by  the  theorem). 

Similarly,  all  the  indirect  effects  come  about  because  the  unaf- 
fected variables  are  correlated  with  the  directly  affected  variable. 
One  might  therefore  think  that  the  unaffected  variable  most  correlated 
with  the  directly  affected  one  will  be  the  unaffected  variable  whose 
coefficient  has  the  biggest  problem.   This  is  clearly  false  as  a  general 
proposition,  since  there  is  no  reason  that  the  ratios  of  the  y     which 
are  multiple  regression  coefficients  must  lie  on  the  same  side  of  unity 
as  the  ratios  of  the  r   ,  the  correlation  or  simple  regression  coefficients. 
Indeed,  it  would  be  odd  if  this  conjecture  were  true,  since  the  grounds 
for  believing  it  are  similar  to,  but  weaker  than  those  for  believing  that  the 
direct  effect  must  be  larger  than  the  indirect  ones. 

Moreover,  it  is  not  true  that  increasing  the  correlation  between 
the  directly  affected  variable  and  a  particular  unaffected  variable  neces- 
sarily worsens  the  inconsistency  in  the  estimate  of  the  latter  variable's 
coefficient.   Thus,  in  the  three-variable  case  in  (10),  an  increase  in 
r^-,  other  things  equal,  can  produce  a  sign  reversal  in  y     and  hence  5  . 

What  is  true  about  an  increase  in  such  a  correlation  is  that  it  has 
an  unambiguously  negative  effect  on  the  inconsistency  involved  (which  may 
Itself  be  positive  or  negative).   To  see  this  (still  in  standard  units), 
differentiate  (4)  with  respect  to  r,,  ^,  obtaining: 

(11)  ^Y^rj^j^_^  =  Plim  {(X(k)'X(k))"^5j^_^} 
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where  e  ^^  is  the  (k-l)-vector  with  last  component  unity  and  other  com- 
ponents zero.  Comparing  this  with  (3),  it  is  evident  that  the  proof  of 
the  theorem  shows: 


^^l^^k-1    =   -y^     j  ^  k-1 


(12)      9  y        .^  r     >   0   ;   /   .  ^  j 

kl   kk  1         ^Vl^^^kk-1 

where  y   is  the  probability  limit  of  the  vector  of  regression  coefficients 
obtained  when  X  ^  is  regressed  on  the  first  k-2  variables.   (Note  that 
this  result  applies  to  any  regression  and  has  nothing  directly  to  do 
with  specification  error.) 
Now, 


kk  kk 

(!)et  Q) 

=  -  ^-^k^-l  "  K   Vl  • 
Thus: 

(14)     3  6^_^/,  r^^.^=U-y^_,S^)/^  r^^_^   =  -  2y2_^  6^  -  6j^0y,_,^  r^^_^)  <  0  , 

from  (12)  and  (13). 

Thus,  the  larger  is  r  ^^ ,  other  things  equal,  the  smaller  (alge- 
braically) will  be  the  inconsistency  in  the  estimate  of  6,  . .  We  have 
already  observed  that  this  may  or  may  not  correspond  to  an  increase  in 

the  absolute  value  of  that  inconsistency.   Does  this  ambiguity  disappear 

2 
as  r,  ,  _.  approaches  unity  so  that  increased  correlation  is  deleterious 

for  coefficients  of  variables  already  very  highly  correlated  with  the 
directly  affected  one? 
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Llke  the  conjecture  that  indirect  effects  are  small  relative  to 
direct  effects,  this  turns  out  to  be  true  only  for  special  cases.   It 
is  obviously  true  for  k  =  2,  or  for  X  .  orthogonal  to  X  , . . .  ,X   . 

Moreover,  when  either  k  =  3  or  the  variables,  1  .  .  .  k-2  are  mutually 

2 
orthogonal,  the  parallel  relation  to  (10)  shows  that  as  r     approaches 

unity,  the  sign  of  6   ^  becomes  that  of  ~r.,_,  so  that,   from  (14),  the 

absolute  value  of  ^,_-,    is  also  growing  (and,  indeed,  will  be  maximized 

at  either  1  or  -1).   For  more  general  cases,  this  is  not  true,  essentially 

for  the  same  sort  of  reasons  as  lie  behind  the  counterexample  to  the 

proposition  that  indirect  effects  are  large  relative  to  direct  ones. 

Similarly,  it  is  not  true  that  an  increase  in  r  ,_^  has  a  bigger  effect 

on  6,  .  than  on  6  ,  j  /  k-1. 

The  failure  of  so  manv  simple  conjectures  is  not  a  matter  for 

depression,  however.   It  is  clear  from  the  theorem  that  the  signs  of 

the  effects  of  simple  specification  error  and,  indeed,  their  relative 

magnitudes  can  be  readily  determined  by  regressing  the  directly  affected 

variable  on  the  remaining  ones.   ^^That  is  necessary  is  to  refrain  from 

behaving  as  though  such  a  multiple  regression  is  equivalent  to  simple  ones. 
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